feat(new source): Add ODBC source implementation - #24044

Summary

This PR implemented a new ODBC(Open Database Connectivity) Source.

Vector configuration

Manual configuration example

[sources.odbc]
type = "odbc"
connection_string = "driver={MySQL ODBC 8.0 ANSI Driver};server=localhost;port=13306;database=vector_db;uid=vector;pwd=vector;"
statement = "SELECT * FROM odbc_table WHERE id > ? LIMIT 1;"
schedule = "*/5 * * * * *"
schedule_timezone = "UTC"
last_run_metadata_path = "odbc_tracking.json"
tracking_columns = ["id"]
statement_init_params = { id = "0" }

[sinks.console]
type = "console"
inputs = ["odbc"]
encoding.codec = "json"

Create example sql file

cat << EOF > example.sql
DROP TABLE IF EXISTS odbc_table;
CREATE TABLE odbc_table
(
    id int auto_increment primary key,
    name varchar(255) null,
    datetime datetime null
);

INSERT INTO odbc_table (name, datetime) VALUES
('test1', now()),
('test2', now()),
('test3', now()),
('test4', now()),
('test5', now());
EOF

Configure ODBC and MariaDB Driver on MacOS

brew install unixodbc
brew install mariadb-connector-odbc

cat << EOF >> /opt/homebrew/etc/odbcinst.ini
[MariaDB Unicode]
Description=MariaDB Connector/ODBC v.3.0
Driver=/opt/homebrew/Cellar/mariadb-connector-odbc/3.2.7/lib/mariadb/libmaodbc.dylib
EOF

Run MariaDB docker container

docker run \
  --rm \
  --name mariadb \
  -e MYSQL_ROOT_PASSWORD=vector \
  -e MYSQL_USER=vector \
  -e MYSQL_PASSWORD=vector \
  -e MYSQL_DATABASE=vector_db \
  -v $(pwd)/example.sql:/docker-entrypoint-initdb.d/example.sql:ro \
  -p 3306:3306 \
  mariadb:latest

How did you test this PR?

I tested it with integration tests of two databases: MariaDB and Postgresql.
(Testing the MySQL container integration in an ARM64 architecture is not simple, so I tested it with MariaDB. Instead, I manually tested MySQL locally.)

cargo vdev integration test odbc-mariadb
cargo vdev integration test odbc-postgresql

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Related: Prototype MySQL Database Source Implementation and Roadmap Inquiry for Official Database Sources/Sinks Support #22931

Notes

• Please read our Vector contributor resources.
• Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
• Some CI checks run only after we manually approve them.
  • We recommend adding a pre-push hook, please see this template.
  • Alternatively, we recommend running the following locally before pushing to the remote branch:
    • make fmt
    • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
    • make test
• After a review is requested, please avoid force pushes to help us review incrementally.
  • Feel free to push as many commits as you want. They will be squashed into one before merging.
  • For example, you can run git merge origin master and git push.
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.